Papers with repeated sampling
Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods conflate fluency with correctness or require substantial computational overhead. |
| Approach: | They propose a single-pass uncertainty quantification method that uses attention matrices to estimate uncertainty without requiring repeated sampling or external models. |
| Outcome: | The proposed method performs well across multiple datasets, task types, and model families and is highly predictive of answer correctness. |
REQUAL-LM: Reliability and Equity through Aggregation in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new method for finding reliable and equitable LLM outputs is developed . REQUAL-LM does not require specialized hardware and does not impose a significant computing load . |
| Approach: | They propose a method for finding reliable and equitable LLM outputs through aggregation. |
| Outcome: | The proposed method minimizes harmful bias while finding reliable outputs . it does not require specialized hardware and does not impose a significant computing load . |
Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Large language models have limited truthfulness and tendency toward overconfidence constrain reliability in factual tasks. |
| Approach: | They propose an efficient method that leverages semantic information encoded in LLMs to quantify uncertainty. |
| Outcome: | The proposed method achieves comparable performance to baselines while significantly reducing computational overhead. |
Keep Guessing? When Considering Inference Scaling, Mind the Baselines (2025.findings-naacl)
Copied to clipboard
| Challenge: | Experiments spanning mathematical reasoning and factual knowledge reveal that repeated sampling improves coverage in large language models. |
| Approach: | They define a baseline that enumerates answers according to their prevalence in the training set and then scale it to measure coverage. |
| Outcome: | The proposed model outperforms repeated model sampling for some LLMs while the coverage for others is on par with that of a mixture strategy that obtains k answers by using only 10 model samples and guessing the remaining k-10 attempts via enumeration. |
RFS-Guard: Detecting Reasoning Hallucinations via Cross-Phase Routing Focus in Large Reasoning Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large reasoning models (LRMs) generate intermediate reasoning traces before the final answer, yet they remain vulnerable to reasoning hallucinations such as subtle arithmetic errors. |
| Approach: | They propose a Routing Focus Score (RFS) that measures how strongly cross-step attention routing aligns with semantic proximity derived from hidden-state cosine similarity. |
| Outcome: | The proposed framework detects and localizes hallucinations without external tools or repeated sampling. |